Systematic, comprehensive, evidence-based approach to identify neuroprotective interventions for motor neuron disease: using systematic reviews to inform expert consensus.

OBJECTIVES: Motor neuron disease (MND) is an incurable progressive neurodegenerative disease with limited treatment options. There is a pressing need for innovation in identifying therapies to take to clinical trial. Here, we detail a systematic and structured evidence-based approach to inform consensus decision making to select the first two drugs for evaluation in Motor Neuron Disease-Systematic Multi-arm Adaptive Randomised Trial (MND-SMART: NCT04302870), an adaptive platform trial. We aim to identify and prioritise candidate drugs which have the best available evidence for efficacy, acceptable safety profiles and are feasible for evaluation within the trial protocol. METHODS: We conducted a two-stage systematic review to identify potential neuroprotective interventions. First, we reviewed clinical studies in MND, Alzheimer's disease, Huntington's disease, Parkinson's disease and multiple sclerosis, identifying drugs described in at least one MND publication or publications in two or more other diseases. We scored and ranked drugs using a metric evaluating safety, efficacy, study size and study quality. In stage two, we reviewed efficacy of drugs in MND animal models, multicellular eukaryotic models and human induced pluripotent stem cell (iPSC) studies. An expert panel reviewed candidate drugs over two shortlisting rounds and a final selection round, considering the systematic review findings, late breaking evidence, mechanistic plausibility, safety, tolerability and feasibility of evaluation in MND-SMART. RESULTS: From the clinical review, we identified 595 interventions. 66 drugs met our drug/disease logic. Of these, 22 drugs with supportive clinical and preclinical evidence were shortlisted at round 1. Seven drugs proceeded to round 2. The panel reached a consensus to evaluate memantine and trazodone as the first two arms of MND-SMART. DISCUSSION: For future drug selection, we will incorporate automation tools, text-mining and machine learning techniques to the systematic reviews and consider data generated from other domains, including high-throughput phenotypic screening of human iPSCs.

Exposure to formaldehyde and asthma outcomes: A systematic review, meta-analysis, and economic assessment.

BACKGROUND: Every major federal regulation in the United States requires an economic analysis estimating its benefits and costs. Benefit-cost analyses related to regulations on formaldehyde exposure have not included asthma in part due to lack of clarity in the strength of the evidence. OBJECTIVES: 1) To conduct a systematic review of evidence regarding human exposure to formaldehyde and diagnosis, signs, symptoms, exacerbations, or other measures of asthma in humans; and 2) quantify the annual economic benefit for decreases in formaldehyde exposure. METHODS: We developed and registered a protocol in PROSPERO (Record ID #38766, CRD 42016038766). We conducted a comprehensive search of articles published up to April 1, 2020. We evaluated potential risk of bias for included studies, identified a subset of studies to combine in a meta-analysis, and rated the overall quality and strength of the evidence. We quantified economics benefit to children from a decrease in formaldehyde exposure using assumptions consistent with EPA's proposed formaldehyde rule. RESULTS: We screened 4,821 total references and identified 150 human studies that met inclusion criteria; of these, we focused on 90 studies reporting asthma status of all participants with quantified measures of formaldehyde directly relevant to our study question. Ten studies were combinable in a meta-analysis for childhood asthma diagnosis and five combinable for exacerbation of childhood asthma (wheezing and shortness of breath). Studies had low to probably-low risk of bias across most domains. A 10-µg/m3 increase in formaldehyde exposure was associated with increased childhood asthma diagnosis (OR = 1.20, 95% CI: [1.02, 1.41]). We also found a positive association with exacerbation of childhood asthma (OR = 1.08, 95% CI: [0.92, 1.28]). The overall quality and strength of the evidence was rated as "moderate" quality and "sufficient" for asthma diagnosis and asthma symptom exacerbation in both children and adults. We estimated that EPA's proposed rule on pressed wood products would result in 2,805 fewer asthma cases and total economic benefit of $210 million annually. CONCLUSION: We concluded there was "sufficient evidence of toxicity" for associations between exposure to formaldehyde and asthma diagnosis and asthma symptoms in both children and adults. Our research documented that when exposures are ubiquitous, excluding health outcomes from benefit-cost analysis can underestimate the true benefits to health from environmental regulations.

Cumulative effects of prenatal-exposure to exogenous chemicals and psychosocial stress on fetal growth: Systematic-review of the human and animal evidence.

BACKGROUND: Adverse effects of prenatal stress or environmental chemical exposures on fetal growth are well described, yet their combined effect remains unclear. OBJECTIVES: To conduct a systematic review on the combined impact and interaction of prenatal exposure to stress and chemicals on developmental outcomes. METHODS: We used the first three steps of the Navigation Guide systematic review. We wrote a protocol, performed a robust literature search to identify relevant animal and human studies and extracted data on developmental outcomes. For the most common outcome (fetal growth), we evaluated risk of bias, calculated effect sizes for main effects of individual and combined exposures, and performed a random effects meta-analysis of those studies reporting on odds of low birthweight (LBW) by smoking and socioeconomic status (SES). RESULTS: We identified 17 human- and 22 animal-studies of combined chemical and stress exposures and fetal growth. Human studies tended to have a lower risk of bias across nine domains. Generally, we found stronger effects for chemicals than stress, and these exposures were associated with reduced fetal growth in the low-stress group and the association was often greater in high stress groups, with limited evidence of effect modification. We found smoking associated with significantly increased odds of LBW, with a greater effect for high stress (low SES; OR 4.75 (2.46-9.16)) compared to low stress (high SES; OR 1.95 (95% CI 1.53-2.48)). Animal studies generally had a high risk of bias with no significant combined effect or effect modification. CONCLUSIONS: We found that despite concern for the combined effects of environmental chemicals and stress, this is still an under-studied topic, though limited available human studies indicate chemical exposures exert stronger effects than stress, and this effect is generally larger in the presence of stress.

Application of the Navigation Guide systematic review methodology to the evidence for developmental and reproductive toxicity of triclosan.

BACKGROUND: There are reports of developmental and reproductive health effects associated with the widely used biocide triclosan. OBJECTIVE: Apply the Navigation Guide systematic review methodology to answer the question: Does exposure to triclosan have adverse effects on human development or reproduction? METHODS: We applied the first 3 steps of the Navigation Guide methodology: 1) Specify a study question, 2) Select the evidence, and 3) Rate quality and strength of the evidence. We developed a protocol, conducted a comprehensive search of the literature, and identified relevant studies using pre-specified criteria. We assessed the number and type of all relevant studies. We evaluated each included study for risk of bias and rated the quality and strength of the evidence for the selected outcomes. We conducted a meta-analysis on a subset of suitable data. RESULTS: We found 4282 potentially relevant records, and 81 records met our inclusion criteria. Of the more than 100 endpoints identified by our search, we focused our evaluation on hormone concentration outcomes, which had the largest human and non-human mammalian data set. Three human studies and 8 studies conducted in rats reported thyroxine levels as outcomes. The rat data were amenable to meta-analysis. Because only one of the human thyroxine studies quantified exposure, we did not conduct a meta-analysis of the human data. Through meta-analysis of the data for rats, we estimated for prenatal exposure a 0.09% (95% CI: -0.20, 0.02) reduction in thyroxine concentration per mg triclosan/kg-bw in fetal and young rats compared to control. For postnatal exposure we estimated a 0.31% (95% CI: -0.38, -0.23) reduction in thyroxine per mg triclosan/kg-bw, also compared to control. Overall, we found low to moderate risk of bias across the human studies and moderate to high risk of bias across the non-human studies, and assigned a "moderate/low" quality rating to the body of evidence for human thyroid hormone alterations and a "moderate" quality rating to the body of evidence for non-human thyroid hormone alterations. CONCLUSION: Based on this application of the Navigation Guide systematic review methodology, we concluded that there was "sufficient" non-human evidence and "inadequate" human evidence of an association between triclosan exposure and thyroxine concentrations, and consequently, triclosan is "possibly toxic" to reproductive and developmental health. Thyroid hormone disruption is an upstream indicator of developmental toxicity. Additional endpoints may be identified as being of equal or greater concern as other data are developed or evaluated.

Correction: Risk of Bias in Reports of In Vivo Research: A Focus for Improvement.

[This corrects the article DOI: 10.1371/journal.pbio.1002273.].

Risk of Bias in Reports of In Vivo Research: A Focus for Improvement.

The reliability of experimental findings depends on the rigour of experimental design. Here we show limited reporting of measures to reduce the risk of bias in a random sample of life sciences publications, significantly lower reporting of randomisation in work published in journals of high impact, and very limited reporting of measures to reduce the risk of bias in publications from leading United Kingdom institutions. Ascertainment of differences between institutions might serve both as a measure of research quality and as a tool for institutional efforts to improve research quality.

Drug repurposing: a systematic approach to evaluate candidate oral neuroprotective interventions for secondary progressive multiple sclerosis.

OBJECTIVE: To develop and implement an evidence based framework to select, from drugs already licenced, candidate oral neuroprotective drugs to be tested in secondary progressive multiple sclerosis. DESIGN: Systematic review of clinical studies of oral putative neuroprotective therapies in MS and four other neurodegenerative diseases with shared pathological features, followed by systematic review and meta-analyses of the in vivo experimental data for those interventions. We presented summary data to an international multi-disciplinary committee, which assessed each drug in turn using pre-specified criteria including consideration of mechanism of action. RESULTS: We identified a short list of fifty-two candidate interventions. After review of all clinical and pre-clinical evidence we identified ibudilast, riluzole, amiloride, pirfenidone, fluoxetine, oxcarbazepine, and the polyunsaturated fatty-acid class (Linoleic Acid, Lipoic acid; Omega-3 fatty acid, Max EPA oil) as lead candidates for clinical evaluation. CONCLUSIONS: We demonstrate a standardised and systematic approach to candidate identification for drug rescue and repurposing trials that can be applied widely to neurodegenerative disorders.

Similar effect of autologous and allogeneic cell therapy for ischemic heart disease: systematic review and meta-analysis of large animal studies.

RATIONALE: In regenerative therapy for ischemic heart disease, use of both autologous and allogeneic stem cells has been investigated. Autologous cell can be applied without immunosuppression, but availability is restricted, and cells have been exposed to risk factors and aging. Allogeneic cell therapy enables preoperative production of potent cell lines and immediate availability of cell products, allowing off-the-shelf therapy. It is unknown which cell source is preferred with regard to improving cardiac function. OBJECTIVE: We performed a meta-analysis of preclinical data of cell therapy for ischemic heart disease. METHODS AND RESULTS: We conducted a systematic literature search to identify publications describing controlled preclinical trials of unmodified stem cell therapy in large animal models of myocardial ischemia. Data from 82 studies involving 1415 animals showed a significant improvement in mean left ventricular ejection fraction in treated compared with control animals (8.3%, 95% confidence interval, 7.1-9.5; P<0.001). Meta-regression revealed a similar difference in left ventricular ejection fraction in autologous (8.8%, 95% confidence interval, 7.3-10.3; n=981) and allogeneic (7.3%, 95% confidence interval, 4.4-10.2, n=331; P=0.3) cell therapies. CONCLUSIONS: Autologous and allogeneic cell therapy for ischemic heart disease show a similar improvement in left ventricular ejection fraction in large animal models of myocardial ischemia, compared with placebo. These results are important for the design of future clinical trials.

Fetal growth and maternal glomerular filtration rate: a systematic review.

OBJECTIVE: Glomerular filtration rate (GFR) may influence concentrations of biomarkers of exposure and their etiologic significance in observational studies of associations between environmental contaminants and fetal growth. It is unknown whether the size of a developing fetus affects maternal GFR such that a small fetus leads to reduced plasma volume expansion (PVE), reduced GFR and subsequent higher concentrations of biomarkers in maternal serum. Our objective was to answer the question: "Is there an association between fetal growth and maternal GFR in humans?" METHODS: We adapted and applied the Navigation Guide systematic review methodology to assess the evidence of an association between fetal growth and GFR, either directly or indirectly via reduction in PVE. RESULTS: We identified 35 relevant studies. We rated 31 human and two non-human observational studies as "low" quality and two experimental non-human studies as "very low" quality. We rated all three evidence streams as "inadequate". The association between fetal growth and GFR was "not classifiable" according to pre-specified definitions. CONCLUSIONS: There is currently insufficient evidence to support the plausibility of a reverse causality hypothesis for associations between exposure to environmental chemicals during pregnancy and fetal growth. Further research would be needed to confirm or disprove this hypothesis.

Systematic review and meta-analysis of interventions tested in animal models of lacunar stroke.

BACKGROUND AND PURPOSE: A total of 25% of strokes are lacunar, and these are pathophysiologically different from large artery strokes. Despite emerging evidence of a substantial impact on physical disability and dementia, little attention has been paid to the development of specific treatments. The optimal use of the animal models of lacunar stroke used to test candidate interventions is not known. METHODS: We conducted a systematic review and meta-analysis of studies testing candidate interventions in animal models of lacunar stroke. We used random-effects meta-analysis to assess the impact of study characteristics and trim and fill to seek evidence of publication bias. RESULTS: The efficacy of 43 distinct interventions was described in 57 publications. The median number of quality checklist items scored was 3 of 8 (interquartile range, 2-4). Many models reflected mechanisms of limited relevance to lacunar stroke. Meta-analysis of results from 27 studies showed that on average, infarct size and neurobehavioral outcome were improved by 34.2% (24.1-44.2) and 0.82 standardized mean difference (0.51-1.14), respectively. Four interventions improved both infarct size and neurobehavioral outcome but there were insufficient data for this finding to be considered robust. For infarct size, efficacy was lower in studies reporting blinding and higher in studies reporting randomization. For neurobehavior, efficacy was lower in randomized studies. For infarct size there was evidence of publication bias. CONCLUSIONS: No intervention has yet been tested in sufficient range and depth to support translation to clinical trial. There is limited reporting of measures to reduce the risk of bias and evidence for a substantial publications bias.

Systematic review and stratified meta-analysis of the efficacy of RhoA and Rho kinase inhibitors in animal models of ischaemic stroke.

BACKGROUND: There is currently only one clinically approved drug, tissue plasminogen activator (tPA), for the treatment of acute ischaemic stroke. The RhoA pathway, including RhoA and its downstream effector Rho kinase (ROCK), has been identified as a possible therapeutic target. Our aim was to assess the impact of study design characteristics and study quality on reported measures of efficacy and to assess for the presence and impact of publication bias. METHODS: We conducted a systematic review and meta-analysis on publications describing the efficacy of RhoA and ROCK inhibitors in animal models of focal cerebral ischaemia where outcome was assessed as a change in lesion size or neurobehavioural score, or both. RESULTS: We identified 25 published papers which met our inclusion criteria. RhoA and ROCK inhibitors reduced lesion size by 37.3% in models of focal cerebral ischaemia (95% CI, 28.6% to 46.0%, 41 comparisons), and reduced neurobehavioural data by 40.5% (33.4% to 47.7%, 30 comparisons). Overall study quality was low (median=4, interquartile range 3-5) and measures to reduce bias were seldom reported. Publication bias was prevalent and associated with a substantial overstatement of efficacy for lesion size. CONCLUSIONS: RhoA and ROCK inhibitors appear to be effective in animal models of stroke. However the low quality score, publication bias and limited number of studies are areas which need attention prior to conducting clinical trials.

Animal models of bone cancer pain: systematic review and meta-analyses.

Pain can significantly decrease the quality of life of patients with advanced cancer. Current treatment strategies often provide inadequate analgesia and unacceptable side effects. Animal models of bone cancer pain are used in the development of novel pharmacological approaches. Here we conducted a systematic review and meta-analysis of publications describing in vivo modelling of bone cancer pain in which behavioural, general health, macroscopic, histological, biochemical, or electrophysiological outcomes were reported and compared to appropriate controls. In all, 150 publications met our inclusion criteria, describing 38 different models of bone cancer pain. Reported methodological quality was low; only 31% of publications reported blinded assessment of outcome, and 11% reported random allocation to group. No publication reported a sample size calculation. Studies that reported measures to reduce bias reported smaller differences in behavioural outcomes between tumour-bearing and control animals, and studies that presented a statement regarding a conflict of interest reported larger differences in behavioural outcomes. Larger differences in behavioural outcomes were reported in female animals, when cancer cells were injected into either the tibia or femur, and when MatLyLu prostate or Lewis Lung cancer cells were used. Mechanical-evoked pain behaviours were most commonly reported; however, the largest difference was observed in spontaneous pain behaviours. In the spinal cord astrocyte activation and increased levels of Substance P receptor internalisation, c-Fos, dynorphin, tumor necrosis factor-α and interleukin-1ß have been reported in bone cancer pain models, suggesting several potential therapeutic targets. However, the translational impact of animal models on clinical pain research could be enhanced by improving methodological quality.

Prevalence and natural history of pain in adults with multiple sclerosis: systematic review and meta-analysis.

The prevalence, associations, and natural history of pain in multiple sclerosis (MS) are poorly understood. The objective of this work was to study the prevalence of pain syndromes in MS both cross-sectionally, and longitudinally during the MS disease course. We systematically identified prospective studies detailing pain prevalence in definite MS. We used pooled prevalence estimates, explored heterogeneity using meta-regression, and analysed prevalence during the disease course using both estimates at disease milestones and longitudinal studies. Twenty-eight articles (7101 subjects) describing overall pain, or pain syndromes, met inclusion criteria. Pooled overall pain prevalence (17 studies, 5319 subjects) was 63% (95% confidence interval [CI] 55-70%). Marked heterogeneity in this estimate was not significantly explained by selected study design variables (use of outpatient sample, timeframe prior to study over which pain was assessed) or sample demographic variables (mean Expanded Disability Status Scale, mean disease duration, proportion of female sex, and proportion with progressive MS). We quantified prevalence of headache (43%; 95% CI 33-52%), neuropathic extremity pain (26%; 95% CI 7-53%), back pain (20%; 95% CI 13-28%), painful spasms (15%; 95% CI 8.5-23%), Lhermitte sign (16%; 95% CI 10-25%), and trigeminal neuralgia (3.8%; 95% CI 2-6%) in included studies. Prevalence of pain at MS disease milestones (prior to onset, at onset, and at relapse) and during longitudinal follow-up was poorly described. Pain is common in MS, as are specific pain syndromes. The clinical associations and natural history of pain in MS require clarification. Future study could be enhanced by standardised study design.

Dopamine agonists in animal models of Parkinson's disease: a systematic review and meta-analysis.

BACKGROUND: Parkinson's disease (PD) can be a severely disabling condition in spite of therapies currently available. Systematic review and meta-analysis can provide an overview of a field of research and identify potential sources of bias and limits to efficacy. In this study we use these tools to describe the reported efficacy of dopamine agonists in animal models of PD. METHODS: Publications were identified by electronic searching of three online databases. Data were extracted for neurobehavioural outcome, for study design and for the reporting of measures to avoid bias. Standardised mean difference meta-analysis was used to provide summary estimates of efficacy, with the effects of study quality and study design explored using stratified meta-analysis. RESULTS: 253 publications reported the use of a dopamine agonist in an animal model of PD; of these 121 reported data suitable for inclusion in meta-analysis. 47 interventions were tested in 601 experiments using 4181 animals. Overall, neurobehavioural outcome was improved by 1.08 standard deviations (SD; 95% Confidence Interval (CI) 0.97-1.19). Reporting of measures to reduce bias was low and publications which reported the blinded assessment of outcome had significantly smaller effect sizes (0.85, 95% CI 0.64 to 1.07) than those which did not (1.18, 95% CI 1.05 to 1.31, p < 0.005). CONCLUSIONS: While dopamine agonists do appear to have efficacy in animal models of PD the low prevalence of reporting of measures to avoid bias is of concern. Systematic review of individual interventions may be helpful in the design of future preclinical and clinical trials.

Systematic survey of the design, statistical analysis, and reporting of studies published in the 2008 volume of the Journal of Cerebral Blood Flow and Metabolism.

Translating experimental findings into clinically effective therapies is one of the major bottlenecks of modern medicine. As this has been particularly true for cerebrovascular research, attention has turned to the quality and validity of experimental cerebrovascular studies. We set out to assess the study design, statistical analyses, and reporting of cerebrovascular research. We assessed all original articles published in the Journal of Cerebral Blood Flow and Metabolism during the year 2008 against a checklist designed to capture the key attributes relating to study design, statistical analyses, and reporting. A total of 156 original publications were included (animal, in vitro, human). Few studies reported a primary research hypothesis, statement of purpose, or measures to safeguard internal validity (such as randomization, blinding, exclusion or inclusion criteria). Many studies lacked sufficient information regarding methods and results to form a reasonable judgment about their validity. In nearly 20% of studies, statistical tests were either not appropriate or information to allow assessment of appropriateness was lacking. This study identifies a number of factors that should be addressed if the quality of research in basic and translational biomedicine is to be improved. We support the widespread implementation of the ARRIVE (Animal Research Reporting In Vivo Experiments) statement for the reporting of experimental studies in biomedicine, for improving training in proper study design and analysis, and that reviewers and editors adopt a more constructively critical approach in the assessment of manuscripts for publication.

Improving the translational hit of experimental treatments in multiple sclerosis.

BACKGROUND: In other neurological diseases, the failure to translate pre-clinical findings to effective clinical treatments has been partially attributed to bias introduced by shortcomings in the design of animal experiments. OBJECTIVES: Here we evaluate published studies of interventions in animal models of multiple sclerosis for methodological design and quality and to identify candidate interventions with the best evidence of efficacy. METHODS: A systematic review of the literature describing experiments testing the effectiveness of interventions in animal models of multiple sclerosis was carried out. Data were extracted for reported study quality and design and for neurobehavioural outcome. Weighted mean difference meta-analysis was used to provide summary estimates of the efficacy for drugs where this was reported in five or more publications. RESULTS: The use of a drug in a pre-clinical multiple sclerosis model was reported in 1152 publications, of which 1117 were experimental autoimmune encephalomyelitis (EAE). For 36 interventions analysed in greater detail, neurobehavioural score was improved by 39.6% (95% CI 34.9-44.2%, p < 0.001). However, few studies reported measures to reduce bias, and those reporting randomization or blinding found significantly smaller effect sizes. CONCLUSIONS: EAE has proven to be a valuable model in elucidating pathogenesis as well as identifying candidate therapies for multiple sclerosis. However, there is an inconsistent application of measures to limit bias that could be addressed by adopting methodological best practice in study design. Our analysis provides an estimate of sample size required for different levels of power in future studies and suggests a number of interventions for which there are substantial animal data supporting efficacy.